Comments for MEDB 5501, Week 8
Review multiple comparisons issue
Type I error: rejecting the null hypothesis when the null hypothesis is true.
Multiple simultaneous hypotheses increase the Type I error rate.
Bonferroni inequality for two simultaneous hypotheses
Define
\(E_1\)
= Type I error for Hypothesis 1
\(E_2\)
= Type I error for Hypothesis 2
\(P[E_1\ \cup\ E_2]=P[E_1]+P[E_2]-P[E_1\ \cap\ E_2]\)
\(P[E_1\ \cup\ E_2]\ \le\ P[E_1]+P[E_2]\)
\(P[E_1\ \cup\ E_2]\ \le\ P[E_1]+P[E_2]\)
\(P[E_1\ \cup\ E_2]\ \le\ 2 \alpha\)
Bonferroni adjustment
For m hypotheses
\(P[E_1\ \cup\ ...\ \cup E_m]\ \le\ m \alpha\)
Test each hypothesis at
\(\alpha/m\)
Preserves overall Type I error rate
Example, 3 simultaneous hypotheses
Reject H0 if p-value < 0.0133
Controversies over Bonferroni adjustment
Increases Type II errors
Impractical for large values of m
Works poorly for highly correlated tests
Ambiguity in definition of “simultaneous” hypotheses
Alternatives to Bonferroni adjustment
False discovery rate
Designation of primary and secondary outcomes
Subjective assessment of simultaneous hypotheses
Review two-sample t-test
\(H_0:\ \mu_1=\mu_2\)
\(H_1:\ \mu_1 \ne \mu_2\)
Or a one tailed alternative
\(T=\frac{\bar X_1-\bar X_2}{SE(\bar X_1-\bar X_2)}\)
Accept H0 if T is close to zero.
What to do with three or more groups?
\(H_0:\ \mu_1=\mu_2=...=\mu_k\)
\(H_1:\ \mu_i \ne \mu_j\)
for some i, j
Note: one-tailed test is tricky.
Accept H0 if the F ratio (defined below) is close to 1.
Important assumptions
Same as independent-samples t-test
Normality
Equal variances
Independence
How to check assumptions
Boxplots
Analysis of residuals,
\(e_{ij}\)
\(e_{ij}\)
= Observed - Predicted
\(e_{ij}= Y_{ij}-\bar{Y}_i\)
Tukey post hoc tests
If you reject H0, which values are unequal
With k groups, there are k(k-1)/2 comparisons
Studentized range (Tukey test)
Requires equal sample sizes per group
Uses harmonic mean approximation for unequal sample sizes.
Do not use harmonic means if seriously different sample sizes.
Alternatives to Tukey post hoc tests
Bonferroni adjustment
Works for unequal sample sizes per group
Works for unequal variances
Dunnett’s test
Treatment versus multiple controls
Scheffe’s test
Works for complex comparison
Example
\(\mu_1\ vs.\ \frac{\mu_2+\mu_3+\mu_4}{3}\)
Artificial data
g y 1 1 23 2 1 30 3 1 25 4 2 33 5 2 36 6 2 41 7 2 37 8 2 43 9 3 24 10 3 29 11 3 31
Scatterplot
Total SS = 452
Within SS = 116
Between SS = 336
Degrees of freedom
For Total SS, df = 10
For Within SS, df = 8
For Between SS, df = 2
In general,
N = number of observations
k = number of groups
Total df = N-1
Within df = N-k
Between df = k-1
ANOVA table, 1 of 4
ANOVA table, 2 of 4
ANOVA table, 3 of 4
ANOVA table, 4 of 4
ANOVA table from the general linear model
Regression approach to ANOVA, 1 of 4
Regression approach to ANOVA, 2 of 4
Regression approach to ANOVA, 3 of 4
Regression approach to ANOVA, 4 of 4
Predicted values and residuals, 1 of 2
Predicted values and residuals, 2 of 2
Recode into different variables dialog box
Old and new Values dialog box
General Linear Model | Univariate dialog box
Descriptives output
Boxplot
Analysis of variance table
Analysis of variance table with irrelevant rows removed
Parameter estimates
Tukey post hoc test, 1 of 7
Tukey post hoc test, 2 of 7
Tukey post hoc test, 3 of 7
Tukey post hoc test, 4 of 7
Tukey post hoc test, 5 of 7
Tukey post hoc test, 6 of 7
Tukey post hoc test, 7 of 7
Checking assumptions, boxplots
Checking assumptions, descriptive statistics
Residual analysis, 1 of 2
Residual analysis, 2 of 2
Analysis of school year, 1 of 4
Analysis of school year, 2 of 4
Analysis of school year, 3 of 4
Analysis of school year, 4 of 4